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IN THE CLAIMS: 

Please amend the claims as follows: 

Claim 1 [[,]]. (currently amended) A method for the characteristics of the rate distortion 
optimization ( RDO) based rate control scheme include comprising : 

Step 1: Bees performing bit allocation for every picture in a GOP which includes an I 
frame, a P frame, or a B frame , and based on the allocated bits a predicted quantization parameter 
being i s used to do a first rate distortion optimization mode selection for every macroblock in the 
current picture|T;l L wherein the predicted quantization parameter is the quantization parameter of 
a previous macroblock, and a coding mode minimizing the following formula is selected as the 
initial coding mode for the current macroblock: 

D(s,c,MODE I QP) + A M0DE R(s,c, MODE I QP) 

wherein s is the luma value of the original macroblock, c is the luma value of the reconstructed 
macroblock, 2.M nnF }s the lagrangian constant: 

for I/P frame. X M0DE = 0.85x2 & ~^: 
for B frame. X M0DE =4x0.85x2 /3 1 

D(s,c,M()DE\QP) is used to evaluate the distortion of the current macroblock after it is 
coded with mode MODE; 

R(s,c,MODE\OP) is the bits used to code the macroblock with mode MODE: 
OP is a quantization parameter for the current macroblock, and is equal to the predicted 
quantization parameter in the first rate distortion mode selection: 

Step 2: T-he the information collected from the first rate distortion mode selection being i s 
used to calculate a final quantization parameter for rate control, and if the final quantization 
parameter is different from the predicted, a second rate distortion mode selection will be 
executed again. 

Claim 2[[,]]. (currently amended) The method of as claim 1 has said , wherein in step 1, 
before coding a GOP, dees bit allocation for the pictures in the GOP with the average picture size 
is performed |T:ll. 

Claim 3[[,]]. (currently amended) The method of as claim 2 has said , wherein the average 
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picture size is calculated as: 

R/F = R+F, here, R is the target bit rate. F is the picture rate. R/F is the average picture 

size. 

Claim 4[[,]]. (currently amended) The method of as- claim 1 and claim 2 have said , further 
comprising performing dees bit allocation adjustment in the coded GOPIT.ll , wherein the T -he 
adjustment is implemented as follows: 



bit _ rate 



N p X p N b X b '8x picture 
K„X. K.X. 



bit _ rate 



N b K p X b ' 8 x picture _ rate 



N p K h X p ' 8 x picture _ rate \ 



here, 77, T p and 77, is the bits allocated to the I, P or B frame respectively[[.]]; Nu N p and Nb is the 
remained none coded I, P or B frames in the GOP respectively[[.]]; Z„ X p and Xb is the global 
complexity estimation for the I, P or B frame respectively and is defined as the multiplier 
between coded bits and average quantization parameter for the frame[[.]]; 
bit_rate is the target bit rate[[.]] i picturejrate is the frame rate[[.]]; 

K p and Kb are constants[[.]] i K p , Kb means the complexity ratio between P, B frame and I 
frame respectively[[.]]; 

R is the remained bits for the GOP, and after coding a picture it is updated as follows: 
7^ = RSi } p } b 

Si }P} b is the coded bits for the current frame. 

Claim 5[[,]]. (currently amended) The method of as- claim 4 has said , wherein before 
coding a GOP, the remaining bits for the current GOP is initialized as follows: 

R = G + Rprev 
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G = bit_rate x N -f picture_rate 
here, is the remained bits for the current GOP[[.]]; 
N is the number of frames in the current GOP[[.]]i 
G is the number of bits for a GOP[[.]]; 

R prev is the remained bits for the previous GOP[[.]L for Per the first GOP, R prev =0. 

Claim 6[[,]]. (currently amended) The method of as- claim 4 has said , wherein X ; > X p and 
Xb are initialized as: 

Xi=axbit_rate 
X p =bxbit_rate 
Xb=cxbit_rate 
here a, b and c are constants[[.]]; 
bitjrate is the target bitrate. 

Claim 7. (canceled). 

Claim 8[[,]]. (currently amended) The method of as claim [[7]] 1 has said , wherein for 
motion estimation in P or B frame, the motion vector minimizing following expression is 
selected as the motion vector for the current macroblock: 

J(m,A M0TI0N ) = SA(T)D(s,c(m)) + A MOTION R(m-p) 
here, D(s,c(m)) is used to evaluate the distortion from motion compensation^.]]; 
SA (T) D is the sum of the absolute difference after prediction (or after Hadmard transform) for 
the macroblock[[.]]; 

R(m-p) is the bits used to code the motion vector[[.]]; 

s is the luma value of the current macroblock in the original frame[[.]]; 

c is the luma value in reference picture[[.]]; 

m is the motion vector[[.]]; 

p is the predicted motion vector[[.]]; 

^motion is the lagrangian constant and A MOT , ON = J~A , MODE [[.]]; 
^mode is the lagrangian constant. 
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Claim 9[[,]]. (currently amended) The method of as claim 2 has said , wherein after the first 

rate distortion mode selection, the RDO based rate control further includes: calculating 

quantization parameter for the current macroblock[[.]]; T-he the quantization parameter is 

adjusted according to the macroblock activity and buffer status. 

Claim 10[[,]]. (currently amended) The method of as- claim 9 has said , wherein the 

quantization parameter for the macroblock is adjusted according to the macroblock activity[[.]]; 

After after the first rate distortion mode selection, the sum of the absolute difference is used as 

the macroblock activity estimation[[.]]; T-he the macroblock activity is calculated as: 

{ixact j)+avg_act 

act = Z\s(i,j)-c(i,j)\ N_act m = - r 

m ■ j act j + (2 x avg _ act ) 

here, i is the horizontal position of the pixel in the current macroblock[[.]]; j is the vertical 
position of the pixel in the current macroblock[[.]]; N_act m is the activity of the current 
macroblock[[.]]; s(i,j) is the luma value of the original pixel(/, j), c(i,j) is the prediction value of 
pixel(i, /)[[.]]; avg_act is the average act m in the previous coded picture which is coded with the 
same type as current picture[[.]]; act m is the sum of the absolute difference after motion 
compensation or intra prediction. 

Claim 11 [[,]]. (currently amended) The method of as claim 9 has said , wherein a virtual 
buffer is used to do rate control[[.]]; First first set up the mapping from the virtual buffer 
occupancy to macroblock quantization parameter, and the final macroblock quantization 
parameter is calculated as: 

f d" x3l) 
Q = \xN act 

d n =d"+B , -T X(m-1)/MB CNT 
m 0 m-1 n ~ 

r = 2xbit _ rate I picture _ rate 

here, Q m is the quantization parameter of current macroblock[[.]]; 

dm is the current buffer occupancy, and it equals dj,dm , and d m h for I, P, B frame 
respectively[[.]]; 

B m .\ is the bits used to code previous macroblock[[.]]; 

do is the initial buffer occupancy for current frameff.]]., n is i,p or b, corresponding to 
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do\d 0 p ,md 4) 

r is the size of virtual buffer. 

Claim 12[[,]]. (currently amended) The method of claim 11, as Claim 11 said, wherein, 
when coding the first frame, the virtual buffer occupancy is initialized with: 



d l Q =10xr/31 

here r is the virtual buffer size; dd,d 0 p ', and do is the initial virtual buffer occupancy for i, 
p, or b frame[[.]]; K p is the complexity ratio between I, P frame; K b is the complexity ratio 
between I, B frame. 

Claim 13[[,]]. (currently amended) The method of claim 12 as claim 2, 9, 10, 11 and 12 
have said , wherein the RDO based rate control also includes a second RDO mode selection, after 
calculating the final quantization parameter for the current macroblock[[.]]; That is to say, thus 
the selected quantization parameter for the current macroblock will be used to do RDO mode 
selection again[[.]]; The the mode which minimizes the following expression will be selected as 
the coding mode for the current macroblock: 

D(s, c,MODE I QP) + A M0DE R(s, c, MODE I QP) 

here, s is the luma value of the original macroblock[[.]] i c is the luma value of the 
reconstructed macroblockff.]],, X M ode'^ the lagrangian constant[[.]]; 

For for W frame, A M0DE = 0. 85 x 2 A ; 

ft,-/ 

For for B frame, A M0DE =4x0.85x2 A [[.]]; 

D(s,c,MODE\QP) is used to evaluate the distortion of the current macroblock coded with 
mode MODE[[.]]; 

R(s,c,MODE\QP) is the bits used to code the macroblock with mode MODE[[.]]± 
QP is the quantization parameter for the current macroblock. 

Claim 14[[,]]. (currently amended) The method of as claim 13 has said , wherein for 
motion estimation in P or B frame, the motion vector minimizing following expression is 
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selected for the current macroblock: 

J(m,X M0TI0N ) = SA(T)D(s,c(m)) + A MOTION R(m - p) 
here, D(s,c(m)) is used to evaluate the distortion from motion compensation^.]]; 
SA (T ) D is the sum of the absolute difference (or after Hadmard transform) for the 
macroblock[[.]]; 

R(m-p) is the bits used to code the motion vector[[.]]; 

s is the luma value of the current macroblock in the original frame[[.]]; 

c is the luma value in reference picture[[.]]; 

m is the motion vector[[.]]; 

p is the predicted motion vector[[.]]; 

^Mor/o/v is the lagrangian constant and A MOTION =^A M0DE [[.]]; 
Amode is the lagrangian constant. 

Claim 15[[,]]. (currently amended) An apparatus for a rate distortion optimization (RDO) 
based rate control implementation includes following modules comprising : a video coding 
encoder module comprising a JVT processing module (for example, H.26 4 encoder module or 
JVT processing module) , rate distortion optimization a RDO mode selection and adaptive 
quantization module, a virtual buffer, and a global complexity estimation module; 

wherein here, the JVT processing module receives the an input frame, and it is connected 
with the RDO mode selection and adaptive quantization module, the virtual buffer module and 
the global complexity estimation module[[.]]; 

the RDO mode selection module and adaptive quantization module is connected with the 
virtual buffer and the global complexity estimation moduleff.JL ft and is configured to receive 
receives the an input signal from the JVT processing module, and processes it to process the 
input signal based on the status of the virtual buffer module and the global complexity module 
stafc»[[,]]i 

bit allocation for every picture in a GOP which includes an I frame, a P frame, or a B 
frame is conducted, a first rate distortion optimization mode selection for every macroblock in 
the current picture using a predicted quantization parameter based on the allocated bits is 
conducted, wherein the predicted quantization parameter is the quantization parameter of a 
previous macroblock, and a coding mode minimizing the following formula as the initial coding 
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mode for the current macroblock is selected: 

D(s,c,MODE I QP) + A MODE R(s,c,MODE I QP) 

where s is the luma value of the original macroblock, c is the luma value of the reconstructed 
macroblock, /U ^n^ is the lagrangian constant; 

for I7P frame, A VCDE = 0.85x2 A - A 
for B frame, A J/C ^ =4x0.85x2 /3 ; 

D(s,cMODE\QP) is used to evaluate the distortion of the current macroblock after it is 
coded with mode MODE; 

R(s,cMODE\OP) is the bits used to code the macroblock with mode MODE; 
QP is the quantization parameter for the current macroblock, and is equal to the predicted 
quantization parameter in the first rate distortion mode selection; 

and the RDO mode selection and adaptive quantization module is configured to calculate a 
final quantization parameter for rate control using the information collected from the first rate 
distortion mode selection, and if the final quantization parameter is different from the predicted, 
execute a second rate distortion mode selection; and then calculate the quantization parameter for 
the macroblock. 

In the last, the JVT processing module wiU is configured to output the final coded 
macroblock with the calculated parameter. 

Claim 16[[,]]. (currently amended) The apparatus of as claim 15 has said , wherein before 
coding a GOP, dees bit allocation for the pictures in the GOP with the average picture size is 
conducted.[[;]] 

Claim 17[[,]]. (currently amended) The apparatus of as claim 16 said, wherein the average 
picture size is calculated as: 

R/F = R+F, here, R is the target bit rate[[.]] i F is the picture rate[[.]] i R/F is the average 
picture size. 

Claim 18[[,]]. (currently amended) The apparatus of claim 17 as claim 16 and 17 have 
said, wherein a dees bit allocation adjustment in the GOPIT.ll is conducted, the T-he adjustment is 
shown as follows: 
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T t = max< 



R bit _ rate 



1 + 



n p X p N 6 X 6 ' 8 X picture _ rate 
K p X t K b X i 




N p K b Xp ' 8 x picture _ rate 



N b K P X b ' 8 x picture _ rate 
K b X p 



R bit _ rate 



R bit_rate 




here, 77, T p and 77, is the bits allocated the to the I, P or B frame respectively[[.]]; N p 
and Nb is the remained none coded I, P or B frames in the GOP respectively[[.]]; X,-, X p and Xb is 
the global complexity estimation for the I, P or B frame respectively and is defined as the 
multiplier between the coded bits and average quantization parameter for the frame[[.]]; 

bit_rate is the target bit rateff.]^ picture_rate is the frame rate[[.]]; 

K p and Kb are constantsff.]^ K p , Kb means the complexity ratio between P,B frame and I 
frame respectively[[.]]; 

R is the remained bits for the GOP, and after coding a picture it is updated as follows: 

R = R-Si, p ,b 

Si }P} b is the coded bits for the current frame. 

Claim 19[[,]]. (currently amended) The apparatus of claim 18, as Claim 18 has said, 
wherein before coding a GOP, the remaining bits for the current GOP is initialized as follows: 



here, R is the remained bits for the current GOP[[.]]; 
N is the number of frames in current GOP[[.]]; 
G is the number of bits for a GOP[[.]]i 

Rprev is the remained bits for the previous GOP[[.]L Per for the first GOP, R prev =0. 

Claim 20[[,]]. (currently amended) The apparatus of as claim 18 said, wherein X,> X p and 



R = G + R, 



G = bitjrate xiV-f picture_rate 
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X b are initialized as: 



Xi=a.xbit_rate 



X p =bxbit_rate 
X b =cxbit_rate 



here a, b and c are constants [[.]]; 
bit_rate is the target bitrate. 

Claim 21. (canceled). 

Claim 22[[,]]. (currently amended) The apparatus of claim 15, as claim 21 has said, 
wherein for motion estimation in P or B frame, the motion vector minimizing following 
expression is selected for the current macroblock: 
J{m,X M0TI0N ) = SA(T)D(s,c(m)) + A M0TI0N R(m -p) 

here, D(s,c(m)) is used to evaluate the distortion from motion compensation^.]]; 

SA ( T ) D is sum of the absolute difference (or after Hadmard transform) for the 

macroblock[[.]]; 

R(m-p) is the bits used to code the motion vector[[.]]; 

s is the luma value of the current macroblock in the original frame[[.]]; 

c is the luma value in reference picture[[.]]; 

m is the motion vector[[.]]; 

p is the predicted motion vector[[.]]; 



^motion is the lagrangian constant and A M0TI0N = ^X MODE [[.]]; 

J-mode is the lagrangian constant. 

Claim 23[[,]]. (currently amended) The apparatus of as- claim 22 has said , wherein after the 
first rate distortion mode selection, the rate control scheme further includes: Calculating 
calculating a new quantization parameter and adjusting it the new quantization parameter 
according to the macroblock activity and buffer status. 

Claim 24[[,]]. (currently amended) The apparatus of as- claim 22 said, wherein for adjusting 
quantization parameter for the current macroblock^.]],, the sum of the absolute difference is used 
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as the macroblock activity estimation after first rate distortion mode selectionlT.il , and the The 
macroblock activity is calculated as: 

(2 x act j ) + avg _act 

act = T\s(i,j)-c(i,j)\ N_act m = - r 

m • ■ act j + (2 x avg _ act) 

here, i is the horizontal position of the pixel in the current macroblocldj.]]^ j is the vertical 
position of the pixel in the current macroblocklT.]]., N_act m is the activity of the current 
macroblocklT.]]., s(i,j) is the luma value of the original pixel(/, j), c(i,j) is the prediction value of 
pixelO', j)[[.]] 3 _avg_act is the average act m in the previous coded picture which is coded with the 
same type as current pictureff.]]^ act m is the sum of the absolute difference after motion 
compensation or intra prediction. 

Claim 25[[,]]. (currently amended) The apparatus of as- claim 22 has said , wherein the [[a]] 
virtual buffer is used to do rate control[[.]]; first First set setting up the mapping from the virtual 
buffer occupancy to macroblock quantization parameterlT.il , the The macroblock quantization 
parameter is calculated as: 

f d" x3l) 
Q m =\— \xN act 

{ r J - - 

r = 2xbit _ rate I picture _ rate 

d n m = d£ + B m _ l -T n x(m-1)/ MB _CNT 

here, Q m is the quantization parameter of current macroblock[[.]]; 

dm is the current buffer occupancy, and it equals d m l ,dm P , and d m b for I, P, B frame 
respectively[[.]]; 

B m .\ is the bits used to code previous macroblock[[.]]; 

do is the initial buffer occupancy for current frame[[.]] i n is i, p or b, corresponding to 
d 0 \do p ,md do h [[.m 

r is the size of virtual buffer occupancy. 

Claim 26[[,]]. (currently amended) The apparatus of as claim 25 said, wherein when 
coding the first frame, the virtual buffer occupancy is initialized with: 

d\ =10xr/31 
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d£=K xd' 
0 p ( 




! 0 



here r is the virtual buffer size; do\do p ', and do is the initial virtual buffer occupancy for i, 
p, or b frame[[.]]; K p is the complexity ratio between I, P frame; K b is the complexity ratio 
between I, B frame. 

Claim 27[[,]]. (currently amended) The apparatus of as claim 23, 2 4 , 25, 26 have said, 
wherein the RDO based rate control also includes a second RDO mode selection, after 
quantization parameter decision for the current macroblock[[.]] i That is to say, thus the decided 
quantization parameter for the current macroblock will be used to do RDO mode selection 
again[[.]]i T-he the mode which minimizes the following expression will be selected as coding 
mode for the current macroblock: 

D{s,c,MODE I QP) + A MODE R(s,c,MODE I QP) 

here, s is the luma value of the original macroblocktt.]^ c is the luma value of the 
reconstructed macroblockff.]^ XmodeIs the lagrangian constant[[.]]; 

for Fef VP frame, X M0DE = 0.85 x l""^ ; 

e-V 

for Fef B frame, X M0DE =4x0.85x2 A [[.]]; 

D(s,c,MODE\QP) is used to evaluate the distortion of the current macroblock after it is 
coded[[.]]; 

R(s,c,MODE\QP) is the bits used to code the macroblock with mode MODE[[.\\i 
QP is the quantization parameter for current macroblock. 

Claim 28[[,]]. (currently amended) The apparatus of as- claim 27 said, wherein for motion 
estimation in P or B frame, the motion vectors minimizes following expression are selected as 
the motion vectors for the current macroblock: 
J(m,A M0TI0N ) = SA(T)D(s,c(m)) + A MOTION R(m -p) 

here, D(s,c(m)) is used to evaluate the distortion from motion compensation^.]]; 

SA ( T ) D is sum of the absolute difference (or after Hadmard transform) for the 

macroblock[[.]]; 
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R(m-p) is the bits used to code the motion vector[[.]]; 

s is the luma value of the current macroblock in the original frame[[.]]; 

c is the luma value in reference picture[[.]]; 

m is the motion vector[[.]]; 

p is the predicted motion vector[[.]]; 

^motion is the lagrangian constant and A MOT1QN = ^]A MODE [[.]]; 
^mode is the lagrangian constant. 

Claim 29[[,]]. (currently amended) The apparatus of claim 28, as Claim 28 said, wherein 
quantization parameter from the RDO mode selection and adaptive quantization module is sent 
back to the JVT processing module, the macroblock is coded by the JVT processing module and 
output of the JVT processing module . 
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